Hand Pose Recognition - Overview and Current Research

Vision-based markerless hand tracking has many applications, for instance in virtual prototyping, navigation in virtual environments, teleand robot-surgery and video games. It is a very challenging task, due to the real-time requirements, 26 degrees-of-freedom, high appearance variability, and frequent self-occlusions. Because of that, and because of the many desirable applications, it has received increasing attention in the computer vision community of the past years. A lot of approaches have been proposed to (partially) solve the problem, but no system has been presented yet that can solve the full-DOF hand pose estimation problem robustly in real-time. The purpose of this article is to present an overview of the approaches that have been presented so far and where future research of hand tracking probably will go. First, we will explain the challenges in more detail. Second, we will classify the approaches; third, we will describe the most important approaches, and finally we will show the future directions and give a short overview of our current work. 1 Motivation and Applications The task of vision-based hand tracking is to estimate the human hand pose based on one or multiple cameras. The scientific interest in this task is very high, and the importance of hand tracking is larger than ever due to the increasing interest in natural user interfaces. The applications for hand tracking are manifold. On the one hand, there are a lot of professional applications such as assembly simulation, motion capture, virtual prototyping, navigation in virtual environments, and rehabilitation. Hand tracking also has a high potential in medical applications, e.g. for sterile interaction with patient related data or tele-surgery. On the other hand, the interest in hand gesture driven game control is increasing strongly. For example, human motion tracking found its way to the consumer market through Nintendo Wii, Sony Move, and Microsoft Kinect. The goal of all three products is to track the human body. The Kinect is the first markerless vision-based consumer product. It is able to track the whole body with fairly high accuracy. The next consequent step is the precise tracking of the human hand, which can significantly improve 2 Daniel Mohr and Gabriel Zachmann